Skip to content

Conversation

@srikrsna-buf
Copy link
Member

@srikrsna-buf srikrsna-buf commented Jun 6, 2025

Switch to Google's CEL implementation.

  • Type checking is stricter, which meant:
    • Converting to CEL types (Long, UnsignedLong, Double, etc...) before passing it to the runtime. Eg: Floats and Double are different for something as simple as - operator and using Floats doesn't work with any function.
    • Changing zero values to CEL supported ones.
    • Declaring separate function overloads based on type. Resulting in a slight increase in function definitions.
  • Functions bindings are based on java types.
  • Partial evaluation is not well documented so it is not implemented in the PR. Also unsure if it is working on main, more on this in a comment.
  • Global variables per program/runtime are not supported. All the variables need to be passed to eval.

Deliberately didn't refactor to keep relatively easier to review.

@srikrsna-buf srikrsna-buf requested review from a user and smaye81 June 6, 2025 07:03
buf("build.buf:buf:${libs.versions.buf.get()}:${osdetector.classifier}@exe")

testImplementation(libs.assertj)
testImplementation(libs.grpc.protobuf)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For google.rpc.Status

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that used?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some of the cel protos we generate code for

Comment on lines +1 to +9
standard_rules/bytes:
- pattern/invalid/not_utf8
# input: [ type.googleapis.com/buf.validate.conformance.cases.BytesPattern ]:{val:"\x99"}
# want: runtime error: value must be valid UTF-8 to apply regexp
# got: validation error (1 violation)
# 1. rule_id: "bytes.pattern"
# message: "value must match regex pattern `^[\\x00-\\x7F]+$`"
# field: "val" elements:{field_number:1 field_name:"val" field_type:TYPE_BYTES}
# rule: "bytes.pattern" elements:{field_number:15 field_name:"bytes" field_type:TYPE_MESSAGE} elements:{field_number:4 field_name:"pattern" field_type:TYPE_STRING}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be solved if we override the bytes_to_string: #304

Comment on lines -141 to -159
Program program = ruleEnv.program(rule.astExpression.ast, globals, PARTIAL_EVAL_OPTIONS);
Program.EvalResult evalResult = program.eval(Activation.emptyActivation());
Val value = evalResult.getVal();
if (value != null) {
Object val = value.value();
if (val instanceof Boolean && value.booleanValue()) {
continue;
}
if (val instanceof String && val.equals("")) {
continue;
}
}
Ast residual = ruleEnv.residualAst(rule.astExpression.ast, evalResult.getEvalDetails());
programs.add(
new CompiledProgram(
ruleEnv.program(residual, globals),
rule.astExpression.source,
rule.rulePath,
ruleValue));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From an internal feature standpoint this is the only step we skipped.

I am also not sure if it ever worked. It seems we have to tell which variables/attrs are not available by passing a PartialActivation type to eval. See here.

@srikrsna-buf srikrsna-buf requested a review from timostamm June 6, 2025 12:25
Copy link
Member

@timostamm timostamm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great so far, but I wasn't able to review everything in this pass.

Comment on lines +26 to +31
/**
* CEL supports protobuf natively but when we pass it field values (like scalars, repeated, and
* maps) it has no way to treat them like a proto message field. This class has methods to convert
* to a cel values.
*/
final class ProtoAdapter {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

* for accessing the variable `now` that's constant within an evaluation.
*/
class NowVariable implements Activation {
class NowVariable implements CelVariableResolver {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, we had a bug early on where now was cached for a validator instance, instead of for a validation pass. The conformance tests can't catch it. Would be good to be sure we don't regress here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not know that. I checked the references for NowVariable class, it is only created here alongside this variable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the issue: #185

buf("build.buf:buf:${libs.versions.buf.get()}:${osdetector.classifier}@exe")

testImplementation(libs.assertj)
testImplementation(libs.grpc.protobuf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that used?

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I think you could also roll #304 into this too, if you wanted, but fine either way.

@srikrsna-buf srikrsna-buf merged commit 3c060a2 into main Jun 6, 2025
4 checks passed
@srikrsna-buf srikrsna-buf deleted the sk/cel branch June 6, 2025 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants